Balancing Cost and Quality: An Exploration of Human-in-the-Loop Frameworks for Automated Short Answer Scoring
نویسندگان
چکیده
Short answer scoring (SAS) is the task of grading short text written by a learner. In recent years, deep-learning-based approaches have substantially improved performance SAS models, but how to guarantee high-quality predictions still remains critical issue when applying such models education field. Towards guaranteeing predictions, we present first study exploring use human-in-the-loop framework for minimizing cost while quality allowing model share with human grader. Specifically, introducing confidence estimation method indicating reliability one can utilizing only high results and casting low graders. our experiments, investigate feasibility proposed using multiple methods datasets. We find that allows automatic graders achieve target quality.
منابع مشابه
development and implementation of an optimized control strategy for induction machine in an electric vehicle
in the area of automotive engineering there is a tendency to more electrification of power train. in this work control of an induction machine for the application of electric vehicle is investigated. through the changing operating point of the machine, adapting the rotor magnetization current seems to be useful to increase the machines efficiency. in the literature there are many approaches wh...
15 صفحه اولEffective Feature Integration for Automated Short Answer Scoring
A major opportunity for NLP to have a realworld impact is in helping educators score student writing, particularly content-based writing (i.e., the task of automated short answer scoring). A major challenge in this enterprise is that scored responses to a particular question (i.e., labeled data) are valuable for modeling but limited in quantity. Additional information from the scoring guideline...
متن کاملthe investigation of the relationship between type a and type b personalities and quality of translation
چکیده ندارد.
An Automated Scoring Tool for Korean Short-Answer Questions Based on Semi-Supervised Learning
Scoring short-answer questions has disadvantages that may take long time to grade and may be an issue on consistency in scoring. To alleviate the disadvantages, automated scoring systems are widely used in America or Europe, but, in Korea, there has been researches regarding the automated scoring. In this paper, we propose an automated scoring tool for Korean short-answer questions using a semi...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Lecture Notes in Computer Science
سال: 2022
ISSN: ['1611-3349', '0302-9743']
DOI: https://doi.org/10.1007/978-3-031-11644-5_38